Lara CarducciYvonne BrownJennifer Leigh MartinJessica PetersonChristina M Prapa
action research - This term is a method of self-improvement of strategies and practices. With this process it allows individuals to identify strategies to address and solve issues. Action Research is important in the classroom because the process allows self-reflection and helps to give examples of how to perform effectively. Action research is data that is observed by individuals on their own practice (Carr et all 1986). In education teachers can use action research to analyze their performance and find strategies to improve delivery skills. applied research - Investigation of 'pure' or basic research findings to determine solutions and development of products, techniques and/or technology. (glossary 615)
The opposite of basic research, applied research is performed to solve problems rather than gather knowledge. Applied research would be used in medical research to search for cures for diseases where basic research would search for more information about the disease itself. The Center For Applied Research in Educational Technology, CARET, “bridges education technology research to practice by offering research-based answers to critical questions”. (CARET, 2005) CARET’s website provides answers to research questions, article reviews, reading and resource lists, news and a glossary of terms. This concept could be applied to a classroom of students with special needs. It has been shown that students with sensory disintegration difficulties benefit from wearing a weighted vest as part of their sensory integration therapy. This helps students stay focused and attentive for longer periods of time.
archival research – Also called secondary research; is a kind of research in which the major base of references and knowledge is from secondary sources such as census data, newspaper clippings, birth certificates and past records in different sectors. (glossary 615)
Taken from the root word archive, this type of research locates, studies and analyzes various documents that have been stored throughout history. One might think that archival research is a daunting task due to the massive amount of recorded information that is available to the public, but that’s not the case. Modern technology has made information much more accessible therefore archival research is easier to perform and sifting through your findings doesn’t take nearly as much time as it had in the past. Archival research is typically done at libraries, county record offices and online. Archival research can be incorporated into many types of lessons, such as requiring students to investigate different time periods throughout history using newspaper headlines and immigration statistics.
case study - This term is an analysis of a variable or research strategy, similar to an experimental analysis. Through case studies, research is analyzed and results are formulated answering “how” and “why” questions. These studies are important in education because the results clarify why situations occur. For example, in a case study on online learning, researchers found that online courses need improvements in order to be accessible for students with disabilities. (Burgstahler et al 2004). In my class room, case studies can be performed to observe and determine which teaching/ learning strategies work best with students.
critical research paradigm - Prevailing pattern of thought in a discipline that provides a whole framework of beliefs, values and methods within which research takes place. (glossary 615)
The word paradigm comes from the Latin word paradeigma meaning show side by side. Manju Basnet says in her blog that “Critical research paradigm is one of the emerging research paradigms in educational research which aims to promote democracy by making changes in different social, political, cultural, economical, ethical as well as other society oriented believes and systems.” (2011) A critical research paradigm is a very directed, structured approach that uses established criteria or steps to perform research. Constructivism is an example of a critical research paradigm that can be used in the classroom. Lessons will be centered around presenting problems that are relevant to the students and starting with “whole” concepts that are broken into “parts”.
field-based research – Includes activities aimed at collecting primary data rather relying on published material. Often involves face-to-face interviewing and direct observation.
The classic form of data collection in naturalistic or field research is observation of participants in the context of a natural scene. Observational data are used for the purpose of description—of settings, activities, people, and the meanings of what is observed from the perspective of the participants. Observation can lead to deeper understandings than interviews alone, because it provides knowledge of the context in which events occur, and may enable the researcher to see things that participants themselves are not aware of, or that they are unwilling to discuss (Patton, 1990). A skilled observer is one who is trained in the process of monitoring both verbal and nonverbal cues, and in the use of concrete, unambiguous, descriptive language.
There are several observation strategies available. In some cases it may be possible and desirable for the researcher to watch from outside, without being observed. Another option is to maintain a passive presence, being as unobtrusive as possible and not interacting with participants. A third strategy is to engage in limited interaction, intervening only when further clarification of actions is needed. Alternatively, the researcher may exercise more active control over the observation, as in the case of a formal interview, to elicit specific types of information. Finally, the researcher may act as a full participant in the situation, with either a hidden or a known identity. Each of these strategies has specific advantages, disadvantages, and concerns that must be carefully examined by the researcher (Schatzman & Strauss, 1973).
Observational techniques are methods by which an individual or individuals gather firsthand data on programs, processes, or behaviors being studied. They provide evaluators with an opportunity to collect data on a wide range of behaviors, to capture a great variety of interactions, and to openly explore the evaluation topic. By directly observing operations and activities, the evaluator can develop a holistic perspective, i.e., an understanding of the context within which the project operates. This may be especially important where it is not the event that is of interest, but rather how that event may fit into, or be impacted by, a sequence of events. Observational approaches also allow the evaluator to learn about things the participants or staff may be unaware of or that they are unwilling or unable to discuss in an interview or focus group. Observations can be useful during both the formative and summative phases of evaluation. For example, during the formative phase, observations can be useful in determining whether or not the project is being delivered and operated as planned. Such formative observations could also provide valuable insights into the teaching styles of the presenters and how they are covering the material.
Focus groups combine elements of both interviewing and participant observation. The focus group session is, indeed, an interview (Patton, 1990) not a discussion group, problem-solving session, or decision-making group. At the same time, focus groups capitalize on group dynamics. The hallmark of focus groups is the explicit use of the group interaction to generate data and insights that would be unlikely to emerge without the interaction found in a group. The technique inherently allows observation of group dynamics, discussion, and firsthand insights into the respondents’ behaviors, attitudes, and language.
Focus groups are a gathering of 8 to 12 people who share some characteristics relevant to the evaluation. Originally used as a market research tool to investigate the appeal of various products, the focus group technique has been adopted by other fields, such as education, as a tool for data gathering on a given topic. Focus groups conducted by experts take place in a focus group facility that includes recording apparatus (audio and/or visual) and an attached room with a one-way mirror for observation. There is an official recorder who may or may not be in the room. Participants are paid for attendance and provided with refreshments. As the focus group technique has been adopted by fields outside of marketing, some of these features, such as payment or refreshment, have been eliminated.
When conducting evaluations, focus groups are useful in answering the same type of questions as in-depth interviews, except in a social context. Specific applications of the focus group method in evaluations include (a) identifying and defining problems in project implementation (b) identifying project strengths, weaknesses, and recommendations (c) assisting with interpretation of quantitative findings (d) obtaining perceptions of project outcomes and impacts, and (e) generating new ideas.
Although focus groups and in-depth interviews share many characteristics, they should not be used interchangeably. An important aspect of conducting focus groups is the topic guide. The topic guide, a list of topics or question areas, serves as a summary statement of the issues and objectives to be covered by the focus group. The topic guide also serves as a road map and as a memory aid for the focus group leader, called a moderator.
informed consent – A legal condition whereby a person can be said to have given consent based upon a clear appreciation and understanding of the facts, implications and future consequences of an action.
The procedures for recording a focus group session are the same as those used for in-depth interviews. However, the focus group approach lends itself to more creative and efficient procedures. If the evaluation team does use a focus group room with a one-way mirror, a colleague can take notes and record observations. An advantage of this approach is that the extra individual is not in the view of participants and, therefore, not interfering with the group process. If a one-way mirror is not a possibility, the moderator may have a colleague present in the room to take notes and to record observations. A major advantage of these approaches is that the recorder focuses on observing and taking notes, while the moderator concentrates on asking questions, facilitating the group interaction, following up on ideas, and making smooth transitions from issue to issue. Furthermore, like observations, focus groups can be videotaped. These approaches allow for confirmation of what was seen and heard. Whatever the approach to gathering detailed data, informed consent is necessary and confidentiality should be assured. Recent advances in computer technology let qualitative researchers rapidly and efficiently gather, enter, and retrieve data such as using e-mail to conduct interviews. Some qualitative researchers take computer notebooks to the field, in which they enter notes directly (Babbie, 1986; Pfaffenberger, 1988). Although many word-processing packages and database managers allow for simple word or phrase searches, specific qualitative analysis programs for text retrieval, such as Ethnograph, ZyIndex, or Word Cruncher, create word lists, count frequency of occurrences, create indexes, and attach key words to words in text (Tesch, 1992).
Personal documents are first-person accounts of events and experiences. These documents include diaries, portfolios, photographs, artwork, schedules, scrapbooks, poetry, letters to the paper, graffiti, object in a home and type of food eaten and clothing worn by the sample. Personal documents can help the evaluator understand how the participant sees the world and what she or he wants to communicate to an audience. In addition, unlike other sources of qualitative data, collecting data from documents is relatively invisible to, and requires minimal cooperation from, persons within the setting being studied (Fetterman, 1989).
The usefulness of existing sources varies depending on whether they are accessible and accurate. Documents can provide the evaluator with useful information about the culture of the institution and participants involved in the project, which in turn can assist in the development of evaluation questions. Information from documents also can be used to generate interview questions or to identify events to be observed. Furthermore, existing records can be useful for making comparisons including comparing project participants to project applicants, project proposal to implementation records, or documentation of institutional policies and program descriptions prior to and following implementation of project interventions and activities.
inter-rater reliability - A reliability process that uses a thematic framework and multiple researchers; designed to provide quality assurance, interpretive accuracy and interpretive authenticity to the interview results (Casey et al, 1996). The following is a scenario on inter-rater reliability provided by Colorado State University (2011):
“Two or more researchers are observing a high school classroom. The class is discussing a movie that they have just viewed as a group. The researchers have a sliding rating scale (1 being most positive, 5 being most negative) with which they are rating the student's oral responses. Inter-rater reliability assesses the consistency of how the rating system is implemented. For example, if one researcher gives a "1" to a student response, while another researcher gives a "5," obviously the inter-rater reliability would be inconsistent. Inter-rater reliability is dependent upon the ability of two or more individuals to be consistent. Training, education and monitoring skills can enhance inter-rater reliability” (Colorado State University, 2011).
The concept of inter-rater reliability may be used in my tenth grade English class when it comes to peer evaluation of student writing. Oftentimes students critique their peers too critically or too generously often experiencing difficulty with giving constructive feedback. I would teach my students how to critique others fairly and provide them with a rubric.
interview analysis coding – Using a codebook of themes that embody the assumptions underlying the analysis, 2 or more researcher conduct data analysis of text (Maxwell, 1996; Miles & Huberman, 1984; Strauss and Corbin, 1990).
Qualitative analysis coding is a process of breaking data into smaller units, determining the import of these units, and putting them back together in an interpreted form. Breaking down the data is a process of classifying or coding. A category is a classification of ideas or concepts. Concepts in the data are examined and compared and connections are made, thus forming categories. Lower-level categories can be organized into higher and more abstract conceptual categories.
Coding is a process for both categorizing qualitative data and for describing the implications and details of these categories. Initially one does open coding, considering the data in minute detail while developing some initial categories. Later, one moves to more selective coding where one systematically codes with respect to a core concept.
Codes are tags or labels assigning units of meaning used to retrieve and organize chunks of information, which cluster segments relating to a particular question, hypothesis, or theme. There are several types of data that can be coded which include observations, intervention, diaries, documents, videos, etc. First-level codes are used to summarize segments of data, and pattern codes are used to group summaries into smaller themes and constructs. Codes can be created using provisional lists, cutting and sorting, creating theme cards, revisions, and conceptual structure. Begin qualitative analysis by coding - review transcripts and documents and identify categories as they emerge. A coding system is developed and refined to facilitate sorting and reviewing data. Then organize by category - - categories result from reading and re-reading field notes, etc. and noting regularities related to ideas, activities, setting, structures, etc. Data are sorted into the categories, resorted, and re-categorized as additional data are collected and analyzed.
An example of coding in a research study is as follows.
Objectives IP-OC Organization IP-ORG Demographics EC-DEM Norms and Authority IC-NORM Critical Events TR-CRIT Internal V-D-I Dependent V-D External V-D-E Evaluation E Effectiveness E-E Positive E-E-P Negative E-E-N Future Use F
The challenges that are faced in developing a coding system are that the coding of texts and in finding the patterns. Coding turns qualitative data (texts) into quantitative data (codes), and those codes can be just as arbitrary as the codes we make up in the construction of questionnaires. Furthermore, the categories one researcher uses to organize qualitative data may not be the same categories another researcher would use to organize the same data. In addition to researcher bias, interest, and style another reason for this is the difference in topics being researched. The topic of the study should have an effect on the categories chosen. Different authors recommend different ways of defining categories. Some suggestions include participant acts, activities, participant meanings, relationships among participants, settings, perspectives of participants, participants’ ways of thinking, regularly occurring activities, infrequently occurring activities, methods, group acts, individual acts, etc. Each of these categories may be further broken down into sub-categories. Relying on pre-defined categories may increase the likelihood that the researcher will miss other important categories in the data. Categories should always be provisional and the researcher should not become too attached to initial categories. The researcher may end up with 100 categories, which must be collapsed into a manageable number - perhaps 10 to 20. Categories are then merged into patterns. Patterns are links among categories that further integrate the data and are used as the basis for reporting. In the end, the researcher has a pyramid of data - - data pieces, categories, and patterns, with each level representing more integration and abstraction. Of course, there is no guarantee that patterns can be induced from the categories.
literature reviews - Surveys scholarly articles, books and other sources (e.g. dissertations, conference proceedings) relevant to a particular issue, area of research, or theory, providing a description, summary, and critical evaluation of each work with the purpose of offering an overview of significant literature published on a topic.
Richard J. Torraco, author of Writing Integrative Literature Reviews: Guidelines and Examples states, “Most integrative literature reviews are intended to address two kinds of topics- mature topics or new, emerging topics” (Torraco, 2005, pg. 1). Furthermore, “ as a topic matures and the size of its literature grows, there is a corresponding growth and development in the knowledge base of the topic. An integrative literature review of a mature topic addresses the need for a review, critique, and the potential reconceptualization of the expanding and more diversified knowledge base of the topic as it continues to develop” (Torraco, 2005, pg. 1). Torraco explains that “ a second kind of integrative literature review addresses new or emerging topics that would benefit from a holistic conceptualization and synthesis of the literature to date” (Torraco, 2005, pg. 2). The concept of the literature review is important to educational research because it is essentially the cornerstone of educational research itself since it is continuously changing with the times on any given subject or topic. Scholars and members of various forms of academia rely on literature reviews for research and writing on scholarly topics. In my tenth grade English class, I would apply the concept of literature reviews in my class by exposing my students to what a literature review is and how it is different from other pieces of literature.
literature search - A systematic and exhaustive search for published material on a specific topic
Just about everyone in today’s world is familiar with how to search Google. However, completing a literature search is a little different. In order to complete a relatively sophisticated search, one would have to typically have access to scholarly published material in order to get the most out of the search. The concept of a literature search is important to educational research because it is constantly changing. In other words, published material is continuously changing and one would have to know how to filter information in order to perform a good literature search. I would teach my tenth grade English class the concept of the literature search by showing them that it is different than doing a search on Google. Perhaps I would introduce them to Boolean logic and have them complete a search on published material on a given topic.
positivist research paradigm – The positivist research paradigm is a “perspective that defines knowledge as something that exists independently in the world and that can be discovered through careful observation; since it exists independently, knowledge is verifiable and stable” (Hinchey, 2008, p. 20). Just like with the scientific method, which is used by natural scientists who follow the positivist model, education researchers who follow the positivist paradigm believe in the “same… method” (p. 21). The positivist model dictates that through observation and structured experiments that can be replicated at any time or place and yield the same result, “universal truths” (p. 21) are discovered. An example of how the positivist research paradigm is used in educational research would be “comparable classes might be identified and some taught a concept in a new, experimental way, others in a traditional way; differences in the test scores might be analyzed to assess the effectiveness of the experimental strategy” (p. 22).
study titles – the title chosen by researchers meant to inform and/or target an audience of readers about the subject of the article. From the time students begin the compose research articles, teachers of any level (from elementary to the collegiate level) can instruct students on how to chose an appropriate title for their study or research that has been conducted. Study titles should catch the reader’s eye without being too outrageous while conveying the correct idea or information about the topic of the article to readers.
triangulation of data - This term is a method used to confirm the results of data. By using several different methods to arrive at one answer, students will build confidence in their answers and work strategies. This method is especially important with solving math equations because it allows students to confirm answers by using many different methods. In my GED class, triangulation of data can help build confidence with math answers because students are able to verify their answers and explain why/how the answer was developed from many angles.
T-tests – A T-Test is a statistical tool which analyses the difference between different samples or populations. “The T-test is one of the most commonly used significance tests to assess whether the means of two groups are statistically significantly different from each other” (Zhang, 2009, p. 25). In education, the T-test can be used in many different occurrences. One example of how I have used a T-test in my own classroom, was when I compared the scores between how my students performed on a spelling test in the beginning of the year (before they were exposed to our school’s spelling program), and how they performed on the same test at the end of the year to see if there was a rate of improvement for the students.
References:
Babbie, E. (1986). Observing ourselves. Belmont, CA: Wadsworth.
Babbie, E. (1989). The practice of social research. Belmont, CA: Wadsworth.
Bogdan R. B. & Biklin, S. K. (1998). Qualitative research for education: An introduction to theory and methods. (3rd ed.). Needham Heights, MA: Allyn and Bacon.
Burgstahler, S., Corrigan,B., & McCarter,J. (2004). Making distance learning courses accessible to students and instructors with disabilities: A case study. The Internet and Higher Education. 7(3), 233-246.
Johnson, S. D. (1995, Spring). Will our research hold up under scrutiny? Journal of Industrial Teacher Education, 32(3), 3-6. Retrieved from http://scholar.lib.vt.edu/ejournals/JITE/
Miles, M. B., and Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. (2nd ed.). Thousand Oaks, CA: Sage Publications.
Miles, M. B., and Huberman, A. M. (2002). The qualitative researcher’s companion. Thousand Oaks, CA: Sage Publications.
Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage Publications,
Pfaffenberger, B. (1988). Microcomputer applications in qualitative research. Newbury Park, CA: Sage Publications.
Robson, C. (2002). Real world research: A resource for social scientists and practitioners. (2nd ed.). Malden, MA: Blackwell Publishing.
Schatzman, L., & Strauss, A. L. (1973). Field research. Englewood Cliffs, N.J.: Prentice-Hall, Inc.
Tesch, R. (1992). Software for qualitative researchers: analysis needs and program capabilities. In N. Fielding (Ed.), Using computers inqualitative research (pp. 16-49). Newbury Park, CA: Sage Publications. Torraco, R .J. (2005). Writing integrative literature reviews: Guidelines and examples. Human Resource Development Review, 4(3), 356-367. Retrieved from ABI/INFORM Global.
Zhang, G. (2009). T-test: The good, the bad, the ugly, & the remedy. Middle Grades Research Journal, 4(2), 25-34. Retrieved from EBSCOhost.
action research - This term is a method of self-improvement of strategies and practices. With this process it allows individuals to identify strategies to address and solve issues. Action Research is important in the classroom because the process allows self-reflection and helps to give examples of how to perform effectively. Action research is data that is observed by individuals on their own practice (Carr et all 1986). In education teachers can use action research to analyze their performance and find strategies to improve delivery skills.
applied research - Investigation of 'pure' or basic research findings to determine solutions and development of products, techniques and/or technology. (glossary 615)
The opposite of basic research, applied research is performed to solve problems rather than gather knowledge. Applied research would be used in medical research to search for cures for diseases where basic research would search for more information about the disease itself. The Center For Applied Research in Educational Technology, CARET, “bridges education technology research to practice by offering research-based answers to critical questions”. (CARET, 2005) CARET’s website provides answers to research questions, article reviews, reading and resource lists, news and a glossary of terms. This concept could be applied to a classroom of students with special needs. It has been shown that students with sensory disintegration difficulties benefit from wearing a weighted vest as part of their sensory integration therapy. This helps students stay focused and attentive for longer periods of time.
archival research – Also called secondary research; is a kind of research in which the major base of references and knowledge is from secondary sources such as census data, newspaper clippings, birth certificates and past records in different sectors. (glossary 615)
Taken from the root word archive, this type of research locates, studies and analyzes various documents that have been stored throughout history. One might think that archival research is a daunting task due to the massive amount of recorded information that is available to the public, but that’s not the case. Modern technology has made information much more accessible therefore archival research is easier to perform and sifting through your findings doesn’t take nearly as much time as it had in the past. Archival research is typically done at libraries, county record offices and online. Archival research can be incorporated into many types of lessons, such as requiring students to investigate different time periods throughout history using newspaper headlines and immigration statistics.
case study - This term is an analysis of a variable or research strategy, similar to an experimental analysis. Through case studies, research is analyzed and results are formulated answering “how” and “why” questions. These studies are important in education because the results clarify why situations occur. For example, in a case study on online learning, researchers found that online courses need improvements in order to be accessible for students with disabilities. (Burgstahler et al 2004). In my class room, case studies can be performed to observe and determine which teaching/ learning strategies work best with students.
critical research paradigm - Prevailing pattern of thought in a discipline that provides a whole framework of beliefs, values and methods within which research takes place. (glossary 615)
The word paradigm comes from the Latin word paradeigma meaning show side by side. Manju Basnet says in her blog that “Critical research paradigm is one of the emerging research paradigms in educational research which aims to promote democracy by making changes in different social, political, cultural, economical, ethical as well as other society oriented believes and systems.” (2011) A critical research paradigm is a very directed, structured approach that uses established criteria or steps to perform research. Constructivism is an example of a critical research paradigm that can be used in the classroom. Lessons will be centered around presenting problems that are relevant to the students and starting with “whole” concepts that are broken into “parts”.
field-based research – Includes activities aimed at collecting primary data rather relying on published material. Often involves face-to-face interviewing and direct observation.
The classic form of data collection in naturalistic or field research is observation of participants in the context of a natural scene. Observational data are used for the purpose of description—of settings, activities, people, and the meanings of what is observed from the perspective of the participants. Observation can lead to deeper understandings than interviews alone, because it provides knowledge of the context in which events occur, and may enable the researcher to see things that participants themselves are not aware of, or that they are unwilling to discuss (Patton, 1990). A skilled observer is one who is trained in the process of monitoring both verbal and nonverbal cues, and in the use of concrete, unambiguous, descriptive language.
There are several observation strategies available. In some cases it may be possible and desirable for the researcher to watch from outside, without being observed. Another option is to maintain a passive presence, being as unobtrusive as possible and not interacting with participants. A third strategy is to engage in limited interaction, intervening only when further clarification of actions is needed. Alternatively, the researcher may exercise more active control over the observation, as in the case of a formal interview, to elicit specific types of information. Finally, the researcher may act as a full participant in the situation, with either a hidden or a known identity. Each of these strategies has specific advantages, disadvantages, and concerns that must be carefully examined by the researcher (Schatzman & Strauss, 1973).
Observational techniques are methods by which an individual or individuals gather firsthand data on programs, processes, or behaviors being studied. They provide evaluators with an opportunity to collect data on a wide range of behaviors, to capture a great variety of interactions, and to openly explore the evaluation topic. By directly observing operations and activities, the evaluator can develop a holistic perspective, i.e., an understanding of the context within which the project operates. This may be especially important where it is not the event that is of interest, but rather how that event may fit into, or be impacted by, a sequence of events. Observational approaches also allow the evaluator to learn about things the participants or staff may be unaware of or that they are unwilling or unable to discuss in an interview or focus group. Observations can be useful during both the formative and summative phases of evaluation. For example, during the formative phase, observations can be useful in determining whether or not the project is being delivered and operated as planned. Such formative observations could also provide valuable insights into the teaching styles of the presenters and how they are covering the material.
Focus groups combine elements of both interviewing and participant observation. The focus group session is, indeed, an interview (Patton, 1990) not a discussion group, problem-solving session, or decision-making group. At the same time, focus groups capitalize on group dynamics. The hallmark of focus groups is the explicit use of the group interaction to generate data and insights that would be unlikely to emerge without the interaction found in a group. The technique inherently allows observation of group dynamics, discussion, and firsthand insights into the respondents’ behaviors, attitudes, and language.
Focus groups are a gathering of 8 to 12 people who share some characteristics relevant to the evaluation. Originally used as a market research tool to investigate the appeal of various products, the focus group technique has been adopted by other fields, such as education, as a tool for data gathering on a given topic. Focus groups conducted by experts take place in a focus group facility that includes recording apparatus (audio and/or visual) and an attached room with a one-way mirror for observation. There is an official recorder who may or may not be in the room. Participants are paid for attendance and provided with refreshments. As the focus group technique has been adopted by fields outside of marketing, some of these features, such as payment or refreshment, have been eliminated.
When conducting evaluations, focus groups are useful in answering the same type of questions as in-depth interviews, except in a social context. Specific applications of the focus group method in evaluations include (a) identifying and defining problems in project implementation (b) identifying project strengths, weaknesses, and recommendations (c) assisting with interpretation of quantitative findings (d) obtaining perceptions of project outcomes and impacts, and (e) generating new ideas.
Although focus groups and in-depth interviews share many characteristics, they should not be used interchangeably. An important aspect of conducting focus groups is the topic guide. The topic guide, a list of topics or question areas, serves as a summary statement of the issues and objectives to be covered by the focus group. The topic guide also serves as a road map and as a memory aid for the focus group leader, called a moderator.
informed consent – A legal condition whereby a person can be said to have given consent based upon a clear appreciation and understanding of the facts, implications and future consequences of an action.
The procedures for recording a focus group session are the same as those used for in-depth interviews. However, the focus group approach lends itself to more creative and efficient procedures. If the evaluation team does use a focus group room with a one-way mirror, a colleague can take notes and record observations. An advantage of this approach is that the extra individual is not in the view of participants and, therefore, not interfering with the group process. If a one-way mirror is not a possibility, the moderator may have a colleague present in the room to take notes and to record observations. A major advantage of these approaches is that the recorder focuses on observing and taking notes, while the moderator concentrates on asking questions, facilitating the group interaction, following up on ideas, and making smooth transitions from issue to issue. Furthermore, like observations, focus groups can be videotaped. These approaches allow for confirmation of what was seen and heard. Whatever the approach to gathering detailed data, informed consent is necessary and confidentiality should be assured. Recent advances in computer technology let qualitative researchers rapidly and efficiently gather, enter, and retrieve data such as using e-mail to conduct interviews. Some qualitative researchers take computer notebooks to the field, in which they enter notes directly (Babbie, 1986; Pfaffenberger, 1988). Although many word-processing packages and database managers allow for simple word or phrase searches, specific qualitative analysis programs for text retrieval, such as Ethnograph, ZyIndex, or Word Cruncher, create word lists, count frequency of occurrences, create indexes, and attach key words to words in text (Tesch, 1992).
Personal documents are first-person accounts of events and experiences. These documents include diaries, portfolios, photographs, artwork, schedules, scrapbooks, poetry, letters to the paper, graffiti, object in a home and type of food eaten and clothing worn by the sample. Personal documents can help the evaluator understand how the participant sees the world and what she or he wants to communicate to an audience. In addition, unlike other sources of qualitative data, collecting data from documents is relatively invisible to, and requires minimal cooperation from, persons within the setting being studied (Fetterman, 1989).
The usefulness of existing sources varies depending on whether they are accessible and accurate. Documents can provide the evaluator with useful information about the culture of the institution and participants involved in the project, which in turn can assist in the development of evaluation questions. Information from documents also can be used to generate interview questions or to identify events to be observed. Furthermore, existing records can be useful for making comparisons including comparing project participants to project applicants, project proposal to implementation records, or documentation of institutional policies and program descriptions prior to and following implementation of project interventions and activities.
inter-rater reliability - A reliability process that uses a thematic framework and multiple researchers; designed to provide quality assurance, interpretive accuracy and interpretive authenticity to the interview results (Casey et al, 1996).
The following is a scenario on inter-rater reliability provided by Colorado State University (2011):
“Two or more researchers are observing a high school classroom. The class is discussing a movie that they have just viewed as a group. The researchers have a sliding rating scale (1 being most positive, 5 being most negative) with which they are rating the student's oral responses. Inter-rater reliability assesses the consistency of how the rating system is implemented. For example, if one researcher gives a "1" to a student response, while another researcher gives a "5," obviously the inter-rater reliability would be inconsistent. Inter-rater reliability is dependent upon the ability of two or more individuals to be consistent. Training, education and monitoring skills can enhance inter-rater reliability” (Colorado State University, 2011).
The concept of inter-rater reliability may be used in my tenth grade English class when it comes to peer evaluation of student writing. Oftentimes students critique their peers too critically or too generously often experiencing difficulty with giving constructive feedback. I would teach my students how to critique others fairly and provide them with a rubric.
interview analysis coding – Using a codebook of themes that embody the assumptions underlying the analysis, 2 or more researcher conduct data analysis of text (Maxwell, 1996; Miles & Huberman, 1984; Strauss and Corbin, 1990).
Qualitative analysis coding is a process of breaking data into smaller units, determining the import of these units, and putting them back together in an interpreted form. Breaking down the data is a process of classifying or coding. A category is a classification of ideas or concepts. Concepts in the data are examined and compared and connections are made, thus forming categories. Lower-level categories can be organized into higher and more abstract conceptual categories.
Coding is a process for both categorizing qualitative data and for describing the implications and details of these categories. Initially one does open coding, considering the data in minute detail while developing some initial categories. Later, one moves to more selective coding where one systematically codes with respect to a core concept.
Codes are tags or labels assigning units of meaning used to retrieve and organize chunks of information, which cluster segments relating to a particular question, hypothesis, or theme. There are several types of data that can be coded which include observations, intervention, diaries, documents, videos, etc. First-level codes are used to summarize segments of data, and pattern codes are used to group summaries into smaller themes and constructs. Codes can be created using provisional lists, cutting and sorting, creating theme cards, revisions, and conceptual structure. Begin qualitative analysis by coding - review transcripts and documents and identify categories as they emerge. A coding system is developed and refined to facilitate sorting and reviewing data. Then organize by category - - categories result from reading and re-reading field notes, etc. and noting regularities related to ideas, activities, setting, structures, etc. Data are sorted into the categories, resorted, and re-categorized as additional data are collected and analyzed.
An example of coding in a research study is as follows.
Objectives IP-OC
Organization IP-ORG
Demographics EC-DEM
Norms and Authority IC-NORM
Critical Events TR-CRIT
Internal V-D-I
Dependent V-D
External V-D-E
Evaluation E
Effectiveness E-E
Positive E-E-P
Negative E-E-N
Future Use F
The challenges that are faced in developing a coding system are that the coding of texts and in finding the patterns. Coding turns qualitative data (texts) into quantitative data (codes), and those codes can be just as arbitrary as the codes we make up in the construction of questionnaires. Furthermore, the categories one researcher uses to organize qualitative data may not be the same categories another researcher would use to organize the same data. In addition to researcher bias, interest, and style another reason for this is the difference in topics being researched. The topic of the study should have an effect on the categories chosen. Different authors recommend different ways of defining categories. Some suggestions include participant acts, activities, participant meanings, relationships among participants, settings, perspectives of participants, participants’ ways of thinking, regularly occurring activities, infrequently occurring activities, methods, group acts, individual acts, etc. Each of these categories may be further broken down into sub-categories. Relying on pre-defined categories may increase the likelihood that the researcher will miss other important categories in the data. Categories should always be provisional and the researcher should not become too attached to initial categories. The researcher may end up with 100 categories, which must be collapsed into a manageable number - perhaps 10 to 20. Categories are then merged into patterns. Patterns are links among categories that further integrate the data and are used as the basis for reporting. In the end, the researcher has a pyramid of data - - data pieces, categories, and patterns, with each level representing more integration and abstraction. Of course, there is no guarantee that patterns can be induced from the categories.
literature reviews - Surveys scholarly articles, books and other sources (e.g. dissertations, conference proceedings) relevant to a particular issue, area of research, or theory, providing a description, summary, and critical evaluation of each work with the purpose of offering an overview of significant literature published on a topic.
Richard J. Torraco, author of Writing Integrative Literature Reviews: Guidelines and Examples states, “Most integrative literature reviews are intended to address two kinds of topics- mature topics or new, emerging topics” (Torraco, 2005, pg. 1). Furthermore, “ as a topic matures and the size of its literature grows, there is a corresponding growth and development in the knowledge base of the topic. An integrative literature review of a mature topic addresses the need for a review, critique, and the potential reconceptualization of the expanding and more diversified knowledge base of the topic as it continues to develop” (Torraco, 2005, pg. 1). Torraco explains that “ a second kind of integrative literature review addresses new or emerging topics that would benefit from a holistic conceptualization and synthesis of the literature to date” (Torraco, 2005, pg. 2). The concept of the literature review is important to educational research because it is essentially the cornerstone of educational research itself since it is continuously changing with the times on any given subject or topic. Scholars and members of various forms of academia rely on literature reviews for research and writing on scholarly topics. In my tenth grade English class, I would apply the concept of literature reviews in my class by exposing my students to what a literature review is and how it is different from other pieces of literature.
literature search - A systematic and exhaustive search for published material on a specific topic
Just about everyone in today’s world is familiar with how to search Google. However, completing a literature search is a little different. In order to complete a relatively sophisticated search, one would have to typically have access to scholarly published material in order to get the most out of the search. The concept of a literature search is important to educational research because it is constantly changing. In other words, published material is continuously changing and one would have to know how to filter information in order to perform a good literature search. I would teach my tenth grade English class the concept of the literature search by showing them that it is different than doing a search on Google. Perhaps I would introduce them to Boolean logic and have them complete a search on published material on a given topic.
positivist research paradigm – The positivist research paradigm is a “perspective that defines knowledge as something that exists independently in the world and that can be discovered through careful observation; since it exists independently, knowledge is verifiable and stable” (Hinchey, 2008, p. 20). Just like with the scientific method, which is used by natural scientists who follow the positivist model, education researchers who follow the positivist paradigm believe in the “same… method” (p. 21). The positivist model dictates that through observation and structured experiments that can be replicated at any time or place and yield the same result, “universal truths” (p. 21) are discovered. An example of how the positivist research paradigm is used in educational research would be “comparable classes might be identified and some taught a concept in a new, experimental way, others in a traditional way; differences in the test scores might be analyzed to assess the effectiveness of the experimental strategy” (p. 22).
study titles – the title chosen by researchers meant to inform and/or target an audience of readers about the subject of the article. From the time students begin the compose research articles, teachers of any level (from elementary to the collegiate level) can instruct students on how to chose an appropriate title for their study or research that has been conducted. Study titles should catch the reader’s eye without being too outrageous while conveying the correct idea or information about the topic of the article to readers.
triangulation of data - This term is a method used to confirm the results of data. By using several different methods to arrive at one answer, students will build confidence in their answers and work strategies. This method is especially important with solving math equations because it allows students to confirm answers by using many different methods. In my GED class, triangulation of data can help build confidence with math answers because students are able to verify their answers and explain why/how the answer was developed from many angles.
T-tests – A T-Test is a statistical tool which analyses the difference between different samples or populations. “The T-test is one of the most commonly used significance tests to assess whether the means of two groups are statistically significantly different from each other” (Zhang, 2009, p. 25). In education, the T-test can be used in many different occurrences. One example of how I have used a T-test in my own classroom, was when I compared the scores between how my students performed on a spelling test in the beginning of the year (before they were exposed to our school’s spelling program), and how they performed on the same test at the end of the year to see if there was a rate of improvement for the students.
References:
Babbie, E. (1986). Observing ourselves. Belmont, CA: Wadsworth.
Babbie, E. (1989). The practice of social research. Belmont, CA: Wadsworth.
Basnet, M. What is critical paradigm? Retrieved July 12, 2007, fromhttp://manju-basnet.blogspot.com/2011/05/what-is-critical-research-paradigm.html
Bogdan R. B. & Biklin, S. K. (1998). Qualitative research for education: An introduction to theory and methods. (3rd ed.). Needham Heights, MA: Allyn and Bacon.
Burgstahler, S., Corrigan,B., & McCarter,J. (2004). Making distance learning courses accessible to students and instructors with disabilities: A case study. The Internet and Higher Education. 7(3), 233-246.
CARET. (2011). Retrieved from http://caret.iste.org/
Carr, W. & Kemmis, S. (1986) Becoming critical: Education, knowledge and action research. London: Falmer Press.
Cooper, D. R. and Schindler, P. S. (2006). Business research methods. (9th ed.). New York, NY: McGraw Hill
Fetterman, D. M. (1989). Ethnography: Step by step applied social research methods series. (Vol. 17). Newbury Park, CA: Sage.
Hinchey, P. (2008). Action research primer. New York, NY: Peter Lang
Howell, J., Miller, P., Park, H.H., Sattler, D., & Schack, T. (2011). Interrater reliability. Retrieved from http://writing.colostate.edu/guides/research/relval/com2a5.cfm
Johnson, S. D. (1995, Spring). Will our research hold up under scrutiny? Journal of Industrial Teacher Education, 32(3), 3-6. Retrieved from http://scholar.lib.vt.edu/ejournals/JITE/
Miles, M. B., and Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. (2nd ed.). Thousand Oaks, CA: Sage Publications.
Miles, M. B., and Huberman, A. M. (2002). The qualitative researcher’s companion. Thousand Oaks, CA: Sage Publications.
Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage Publications,
Pfaffenberger, B. (1988). Microcomputer applications in qualitative research. Newbury Park, CA: Sage Publications.
Robson, C. (2002). Real world research: A resource for social scientists and practitioners. (2nd ed.). Malden, MA: Blackwell Publishing.
Schatzman, L., & Strauss, A. L. (1973). Field research. Englewood Cliffs, N.J.: Prentice-Hall, Inc.
Seidel, J. V. (1998). Qualitative data analysis. Retrieved from http://ritagoh.com/Academia/Qualitative%20Data%20Analysis.pdf
Tesch, R. (1992). Software for qualitative researchers: analysis needs and program capabilities. In N. Fielding (Ed.), Using computers in qualitative research (pp. 16-49). Newbury Park, CA: Sage Publications.
Torraco, R .J. (2005). Writing integrative literature reviews: Guidelines and examples. Human Resource Development Review, 4(3), 356-367. Retrieved from ABI/INFORM Global.
Zhang, G. (2009). T-test: The good, the bad, the ugly, & the remedy. Middle Grades Research Journal, 4(2), 25-34. Retrieved from EBSCOhost.